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Executive Summary 



N ew research conducted by Boston’s Center for Collaborative Education 
documents significant achievement by students who attend the city’s 
Pilot Schools. Pilot School students are performing better than the district 
averages across every indicator of student engagement and performance, 
including the statewide standardized assessment (MCAS). In other standard 
measures, Pilot School students show better rates of attendance and fewer out- 
of-school suspensions, and more go on to attend university or technical college 
after they graduate. 



Key findings of the report 1 

Academic Performance 

• Higher performance on the state standardized test — the 
Massachusetts Comprehensive Assessment System (MCAS) 

When MCAS scores of students at Pilot Schools are compared to BPS 
averages, Pilot School students consistently score better. For example, 46 
percent of the Pilot School fourth graders were rated advanced or proficient in 
the grade 4 English Language Arts (ELA) MCAS, compared to 29 percent of 
BPS students. Math scores were also far apart, with 37 percent of Pilot School 
fourth-grade students scoring advanced or proficient, compared to 21 percent 
of BPS students. The percentage of fourth-grade students that received passing 
scores in the math and ELA MCAS tests was also higher in Pilot Schools, 
although the difference was smaller. 



1 The study reports on 15 of the 19 schools, leaving out the Early Learning Center, 2 schools that opened in 
the year studied, and another school that opened in 2004-2005. 
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Our vision is to help every student, when they leave us, to not only 
get to college, but to get them through college. — Principal 



The tenth-grade English Language Arts test showed a particularly wide 
disparity between Pilot School students and BPS students. On average, 84 
percent of Pilot School students passed the test, compared to 58 percent of BPS 
students. As a proportion, more than twice as many Pilot School students as 
BPS students were in the advanced/proficient category. 

Tenth-grade math MCAS scores were similarly higher than those posted by 
BPS high schools. Eighty percent of Pilot School students passed the tenth- 
grade math MCAS test, compared to 59 percent of BPS students. 

It is noteworthy that the difference in performance between Pilots and BPS 
schools increased by grade level — so that the largest differences in MCAS 
performance are at the high school level. This difference is also seen in the 
engagement indicators of attendance and suspension levels discussed below. 

• Higher college-going rate 

A higher proportion of Pilot School students continue their education at a 
university or a technical college, as compared to BPS graduates. 2 The average 
for all Pilot Schools is 79 percent, compared to an average of 67 percent for all 
BPS graduates. Individual Pilot Schools did far better than the average. For 
example, 94 percent of students graduating from Fenway High School, a Pilot 
School, attended university or technical college. One Pilot School, Greater 
Egleston Community High School, posted a percentage of students going on to 
university or technical college that was lower than the BPS average — 50 
percent compared to 67 percent. However, that reflects in part Greater 
Egleston’s mission, which is to serve students who have previously left or 
dropped out of other high schools. 



I like the [idea] of this school 
being a college prep [school]. 
We have a lot more discussions 
in the classroom, and the 
teachers involve every 
student. — Student 



Engagement 

• Higher attendance rates 

• Lower suspension rates 

• Lower in-district and out-of-district transfer rates 



For attendance, elementary Pilot Schools posted a median rate of 97 percent, 
compared to 96 percent for BPS elementary schools. Pilot middle schools 
posted a median attendance rate of 97 percent, compared to 94 percent in BPS 
middle schools. And for high schools, the Pilot School median attendance rate 
was 95 percent, compared to a BPS high school rate of 89 percent. 



2 Survey data on post-secondary participation of 2003 high school graduates was provided by the Boston 
Private Industry Council (PIC) as derived population estimates. Results are displayed as the proportion of 
graduates who were enrolled in post-secondary education one year after graduation. 
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Executive 

Summary 



For out-of-school suspension rates, BPS elementary schools posted an 
average of 3 percent, compared to 1 percent for Pilot Schools. Among middle 
schools, BPS schools posted an average of 14 percent, compared to 12 percent 
for Pilot Schools. And for high schools, BPS had a rate of 9 percent out-of- 
school suspensions compared to 5 percent for Pilot Schools. 

Boston School Superintendent Thomas Payzant says he is encouraged by 
the results of the study. “What this report shows is that real progress that can 
change the lives of students is possible. Pilot Schools have made an invaluable 
contribution to public education in Boston.” The Pilot Schools have grown 
and thrived under Payzant’s ten-year stewardship. Payzant points out that 
many other school systems around the country continue to send observers to 
Boston to see Pilot Schools, an important part of the Boston Public Schools’ 
comprehensive pre-K through 12 reform plan. 

Richard Stutman, president of the Boston Teachers Union, is also proud of 
the progress Pilot Schools have made. “This report shows that the Pilot model is 
a vital avenue for teacher growth and innovation. It is important for us that the 
lessons and best practices learned from Pilots be considered for other schools. 

It’s exciting to see the progress that has been made by Boston students.” 

Pilot Schools are Boston Public Schools 

The Pilot Schools — which operate with autonomy within the school district — 
were created in 1995 through a unique partnership that included the mayor, 
the office of the superintendent, the school committee, and the teachers union. 
Boston is the only city in the country to create Pilot Schools to serve as models 
of innovation, with the purpose of identifying “best practices” and sharing 
them with public school educators in Boston and beyond. In addition to 
educating the children who attend them, the Pilot Schools serve as research 
and development laboratories, creating and assessing strategies that can create 
success within an urban public school system. 

Students in Pilot Schools are on the whole representative of students in the 
public system with regard to economic status; race and ethnicity; and in the 
proportion of mainstream special education students attending the schools. ’ As 
well, over time Pilot Schools are becoming increasingly more representative of 
students with moderate to severe special needs. The number of Pilot Schools 
has grown from the first year, when 5 Pilot Schools enrolled about 1.5 percent 
of the BPS population. Today, 19 schools (pre-K through 12) in Boston use the 
Pilot School model to serve 5,900 students, or about 10 percent of the public 
school population. 4 By creating the Pilot Schools — which offer choice, 
smallness, and accountability — the Boston Public Schools have taken on a 
national leadership role in urban school reform. 



3 A primary goal of the Pilot School Network is to serve students representative of BPS as a whole. Pilot 
middle and high schools have achieved this goal. Representative enrollment has not yet been reached at the 
elementary level — Pilot elementary schools serve a smaller percent of low-income students and students of 
color than BPS. However, as new elementary Pilot Schools have opened, Pilot elementary demographics are 
getting closer to BPS elementary student demographics. 

4 Two of the 19 are Horace Mann charter schools in addition to being part of the Pilot School Network. 
Horace Mann charters are granted autonomy by the state department of education while also remaining part 
of the district and the teachers union. 
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Pilot Schools have used their autonomy to create curriculum, 
assessment, and school structures that support high expectations 
and achievement. 



Key features of Pilot Schools 

Although they serve essentially the same student population, Pilot Schools are 
different from traditional public schools in significant ways. Pilot Schools have 
far more autonomy over their resources (including budget, staffing, curriculum, 
governance, and the calendar) in order to best serve their students. While Pilot 
Schools employ a diversity of educational approaches, they share certain key 
characteristics. Pilot Schools are: 



• Accountable. Pilot Schools are held to high standards of performance 
through a five-year-cycle, high-stakes school quality review process, using 
a set of benchmarks that articulates the criteria for high-performing 

schools. Five Autonomies: 



• Small and Nurturing. Pilot Schools place great emphasis on creating a 
nurturing school culture in which teachers attend closely to each student’s 
learning needs. Only two of the 19 Pilot Schools have more than 500 
students, and both of them are organized into multiple small academies. 



• Budget 

• Staffing 

• Curriculum, Instruction, 
and Assessment 



• Vision-Driven. Every Pilot School has created a vision focused on 
equity and the fundamental belief in every child’s potential. This vision 
shapes the teaching and the work of every member of the school 
community. Pilot Schools have the power then to hire teachers and staff 
who can support this vision. 



• Governance and Policies 

• Schedule 



Pilot Schools use their autonomy to improve teaching 
and learning 

The report emphasizes that Pilot Schools have used their autonomy to create 
curriculum, assessment, and school structures that support high expectations 
and achievement. Pilot Schools commit to making time for faculty collaboration 
and planning — which are crucial for improving a school’s culture and 
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Current Boston 
Pilot Schools 




School Name 


Grades 
Served in 
2003-2004 


Elementary 

Baldwin ELC 


KOto 1 


Samuel Mason School 


KO to 5 


Lee Academy 


N/A 


Elementarv-Middle 


Lyndon Elementary 
School 


K to 8 


Young Achievers 


K to 8 


Mission Hill School 


K to 8 


Orchard Gardens 
Pilot School 


K to 8 


Middle 

The Harbor School 


6 to 8 


Lilia G. Frederick 
Pilot Middle School 
(formerly New Boston 
Pilot Middle School) 


6 to 8 


Middle-High 

Josiah Quincy 
Upper School 


6 to 10 


High 

Fenway High School 


9 to 12 


Greater Egleston 
Community 
High School 


ungraded 


New Mission 
High School 


9 to 12 


Health Careers Academy 
(Horace Mann Charter) 9 to 1 2 


Boston Arts Academy 


9 to 12 


Boston Day and 
Evening Academy 
(Horace Mann Charter) 


ungraded 


Boston Community 
Leadership Academy 
(BCLA) 


9 to 12 


Tech Boston Academy 


9 to 10 


Another Course 
to College 


9, 1 1, 12 






performance. With the same per pupil budget as BPS schools, Pilot Schools as 
compared to the district average have: 

• Low class sizes (average of 20 in elementary schools and 19 in 
secondary schools). 

• Low overall student-teacher loads in secondary schools (average 
of 55 students per teacher). 

• Long instructional periods. 

• Significant collaborative faculty planning time, a key correlate to 
increased student achievement. 

• A nurturing school culture, featuring advisories, learning centers, and 
student support teams. 

• Graduation by demonstrating competency or mastery. In almost 
all Pilot Schools, in order to graduate (from eighth or twelfth grades), 
students must demonstrate mastery of a defined set of skills and content 
knowledge through a series of assessments. This approach stands in 
contrast to the traditional standard of qualifying for graduation by 
course completion and adequate performance on standardized tests. 

The Pilot School strategy has strengthened the Boston 
Public Schools 

Excellent single schools are relatively common, but it is rare for excellence to 
spread throughout a large urban district. The Pilot School strategy has also led 
to improvements throughout the Boston Public Schools: 

• The Pilot approach to high school graduation was instrumental 
in leading to a new BPS graduation policy. Under the new policy, a 
high school can choose to propose a unique course sequence and 
assessment for graduation. 

• The success of the Pilot high schools was an important factor in 
Superintendent Payzant’s recommendation to the school committee 
to authorize conversion of four large BPS high schools to small schools 
sharing space and resources. These new small schools have been granted 
limited autonomy over their budgets. 

• The Pilot autonomies have informed the use of autonomy in other 
selected BPS schools. These schools can seek autonomy from a district 
policy if the waiver will advance their teaching and learning agendas. 

Through their choice, commitment, and hard work, Pilot School teachers, 
students, and families have created models of educational excellence and 
innovation within the BPS system. They demonstrate that teacher unions and 
districts working together can make the system stronger and more successful. 
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W hat does it take to give each student access to an excellent education? 

Boston High School was once one of the lowest-performing high 
schools in the Boston Public Schools. When the school department announced 
in 2001 that it would close the school, the principal led the staff, students, and 
families through a process that would keep it open. The superintendent 
allowed the school to propose conversion to Pilot status, and in September 
2002, Boston High School opened as Boston Community Leadership Academy 
(BCLA), a Pilot high school. The Pilot high school continued to serve a 
demographically similar population of students while it became intentionally 
smaller through graduation and smaller incoming classes. 

In two years, the school achieved dramatic improvement in student 
engagement and achievement. BCLA was able to accomplish this success by 
making significant changes in mission and vision, schedule, staffing, 
curriculum, and leadership roles — changes that would not have been possible 
without the autonomy granted through Pilot status. Progress was not easy. 
Many of Boston High’s students faced being part of a school that was not like 
the high school they had chosen to attend. As one student put it, “At first, we 
weren’t really aware of all the changes... we thought the school’s title was just 
going to change, but we didn’t realize that it was going to get a lot more 
academically rigorous... I think... looking back, it was a great change. In the 
beginning, I was a little apprehensive... but it got better. ..a lot better.” 

Within two years of its conversion to Pilot status, BCLA surpassed the 
district average in both math and English Language Arts (ELA) performance 
on MCAS. In 2001, Boston High students performed below the district in both 
math and ELA, with 31% of students passing math and 41% passing ELA. In 
2004, more than three quarters of students passed the math and ELA tests. 
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As we examine BCLA and the other Boston Pilot Schools — in terms of 
student demographics, engagement, and performance — behind the data is the 
work of thousands of students, families, and teachers. Their accomplishments, 
shown in this report, have strengthened Boston’s public school system. The 
Pilot School model promises to inform work in other districts as well. 



The Purpose of Pilot Schools 

The Boston Pilot Schools were explicitly created to be models of educational 
innovation and to serve as research and development sites for effective urban 
public schools within the district. The result of a partnership among the 
Boston mayor, school committee, superintendent, and teachers union, the 
Boston Pilot Schools were opened in 1995 to promote increased choice options 
within the school district, largely in response to 1994 state legislation creating 
first-time charter schools and the potential subsequent loss of Boston students 
to area charter schools. The Pilot Schools Network is a systemic model that 
demonstrates how districts and teachers unions can work together to support 
the creation of innovative schools. 

What makes this network of public schools unique is that, by virtue of an 
innovative teachers union contract, and by stipulation of the school 
department, they have autonomy over budget, staffing, governance, 
curriculum, and schedule (BTU, 2003). These autonomies provide increased 
flexibility to organize schools and staffing to best meet students’ needs, while 
operating within the economies of scale of a large urban public school district. 
Teachers tend to cite curriculum autonomy as a crucial component of Pilot 

status. Administrators credit their 
budget and staffing autonomies 
with allowing them to shape the 
culture of the school. As one 
principal puts it: “We align our 
resources with our student 
needs. [We put] more focus on 
teaching and learning and less 
focus on ‘administrivia.’ As our 
school climate is becoming 
healthier, we are spending less 
money on discipline....” 

As part of a network, Pilot 
Schools are not isolated entities. The 

Center for Collaborative Education (CCE), a nonprofit education organization, 
convenes the Pilot Schools Network and provides the Pilot Schools with 
coaching services, professional development, advocacy, and research. These 
network activities, facilitated by CCE, allow the Pilot Schools to collaborate 
with one another in pursuit of their goals and to create mutual accountability 
for results. 



Through their choice, commitment, and hard work, Pilot 
School teachers, students, and families have created 
models of educational excellence and innovation within 
the Boston Public School system. They have made the 
system stronger and more successful. 
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We align our resources with our student needs. [We put] more 
focus on teaching and learning and less focus on ‘administrivia.’ 

As our school climate is becoming healthier, we are spending less 
money on discipline.... — Pilot principal 



Key Features of Pilot Schools 

Pilot Schools are: 

• Autonomous. The defining philosophy of Pilot Schools is that if schools 
are provided maximum control over their resources to create an 
innovative education program, in exchange for increased accountability, 
student engagement and performance will improve. Pilot Schools have 
autonomy over budget, staffing, curriculum, governance, and schedule. 

• Accountable. Pilot Schools are held to high standards of performance 
through a five-year-cycle, high-stakes school quality review process, using a 
set of benchmarks that articulate the criteria for high-performing schools. 

• Small and Nurturing. Pilot Schools are small, personalized, and 
democratic, enrolling 500 students or fewer, enabling adults to know 
students well. Two schools enrolling more than 500 students are 
organized into multiple small academies. Every school places great 
emphasis on creating a nurturing school culture in which staff pay close 
attention to each student’s learning needs. 

• Vision-Driven. Every Pilot School has an articulated vision of 
successfully educating all of its students, with teaching and learning at 
the vision’s core. Pilot Schools have the latitude to hire staff who are 
committed to fulfilling the school’s vision. 

• Focused on Equity. Pilot Schools embrace as a core belief the potential 
of every student to achieve academic success, regardless of his or her 
background and past educational experience. Pilot Schools are not 
selective. The Network strives to enroll students representative of the 
larger district. 
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Pilot Schools are 

• Autonomous 

• Accountable 

• Small and Nurturing 

• Vision-Driven 

• Focused on Equity 



Expansion of Pilot Schools Network 

The number of Pilot Schools has increased considerably since they first were 
opened, when there were five Pilot Schools enrolling fewer than 900 students. 
Currently, the Network includes 19 Pilot Schools, two of which are also 
Horace Mann schools, 5 spanning grades preK-12 and serving approximately 
5,900 students, or about 10% of the total Boston Public Schools (BPS) 
enrollment. 

The table on page 7 lists each Pilot School, the grades it serves, its initial 
year of Pilot status, and how it became a Pilot School. 

The graphs below reflect that, even in years in which new Pilot Schools did 
not open, enrollments increased, because there were schools adding one grade 
at a time. 

The growth in the number of Pilot Schools and the schools’ encouraging 
results have led to a great deal of interest in establishing more of these schools. 
In the fall of 2002, more than 30 regular BPS schools attended a grant 
orientation meeting sponsored by The Boston Foundation to learn more about 
the possibility of receiving seed grants from the foundation to explore the 
benefits of converting to Pilot status for their own school. Ultimately, 13 BPS 
schools received these planning grants, and five of the 13 schools voted to 
convert to Pilot status — a watershed moment for the Boston Public Schools. 



Pilot Schools Network Expansion 
Number of Pilot Schools 




Pilot Schools Network Expansion 
Number of Pilot School Students 6 




5 The two Boston Horace Mann Charter Schools are also Pilot Schools. Horace Mann Charter Schools are 
granted autonomy by the state department of education while also remaining part of the district and the 
teachers union. 

6 Data for this graph were downloaded from the National Center for Education Statistics (NCES) Common 
Core of Data (http://nces.ed.gov/ccd/). 



4 Progress and Promise: Results from the Boston Pilot Schools 



www.cce.org 



The Pilot Schools encompass a diversity of educational approaches tailored 
to meet particular student needs and interests. They include schools focused on 
the arts, technology, and leadership, serving students in the following grade 
levels: early childhood, K-8, elementary, middle, and high schools; and 
schools focused on the needs of older students who have not achieved success 
in mainstream schools. Pilot Schools offer strong choices to families. A 2003 
CCE study found that high school students and their families choose Pilot 
Schools for their academic challenge, safe and caring environments, and 
specialty offerings (Doyle et al, 2003). 




How Pilot Schools Fit into the National and Local 
School Reform Landscape 

Public schools in the United States are at a critical juncture, with the No Child 
Left Behind (NCLB) Act placing an enormous amount of pressure on schools 
to improve student performance as measured by standardized tests. At the 
same time, a burgeoning small schools movement — supported in large part by 
the Bill & Melinda Gates Foundation — is focused on improving the 
achievement of low-income students and students of color, and particularly on 
closing gaps in achievement between these subgroups and white or more 
affluent students. Simultaneously, a national focus on providing families with 
greater choice in public education has given rise to charter schools and 
vouchers. These two strategies have gained momentum over the past decade, 
even as they generate a great deal of controversy due to their pressure on 
funding for public school districts. 

The Boston Public Schools have taken on a national leadership role in 
urban school reform by creating the Pilot Schools, which offer choice, 
smallness, and accountability without draining resources from the public 
school system. The Pilot School strategy has contributed to the district in 
significant ways. The Pilot model of competency-based graduation was 
instrumental in leading to a new BPS graduation policy, in which every high 
school has the option to propose a unique course sequence and assessments for 
graduation that meet or exceed the district standards. More broadly, the Pilot 
autonomies have informed the creation of “Strategic Planning Schools” within 
the district. Any of these schools can seek autonomy from a district policy if 
the waiver will advance its teaching and learning agenda. Finally, the success 
of the Pilot high schools informed Superintendent Payzant’s proposal to the 
school committee to convert four large, comprehensive BPS high schools to 13 
small schools sharing space and resources. These new small schools have been 
granted limited autonomy over their budgets. 



The Boston Public Schools 
have taken on a national 
leadership role in urban 
school reform by creating the 
Pilot Schools, which offer 
choice, smallness, and 
accountability 
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Because of their unique role as laboratories of educational 
innovation, it is important to understand how well the Pilot Schools 
are doing in comparison with other schools in the district. 



Why Examine the Pilot Schools’ Outcomes? 

Critical examination of progress across a range of indicators is still rare in 
public education. Test scores alone do not offer a complete picture of a 
school’s success. This report presents data which answer the following research 
questions: 

1. How demographically representative of the Boston Public Schools are 
Pilot School students? 

2. What are some unique characteristics of Pilot Schools that help them to 
organize for high-quality teaching and learning? 

3. What are the Pilot School outcomes for student engagement indicators 
(as measured by attendance, out-of-school suspensions, district leavers, 
and in-district transfers)? 

4. What are the Pilot School outcomes for student performance indicators 
(as measured by MCAS, grade-level retentions, college plans, and college 
enrollment)? 

Because of their unique role as laboratories of educational innovation, it is 
important to understand how well the Pilot Schools are doing in comparison 
with other schools in the district. A rich picture emerges, one that will likely 
spark discussions at all levels of a community. 
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School Name 


Grades 
Served 
in 2003-04 


Initial Year 
of Pilot 
Status 


How School 
Became a 
Pilot School 


Elementary 








Baldwin Early Learning Center 


KOto 1 


2003 


converted 


Samuel Mason School 


K0 to 5 


2003 


converted 


Lee Academy 


NA 7 


2004 


opened 


Elementary-Middle 








Lyndon Elementary School 


K to 8 


1995 


opened 


Young Achievers 


K to 8 


1995 


opened 


Mission Hill School 


K to 8 


1997 


opened 


Orchard Gardens Pilot School 


K to 8 


2003 


opened 


Middle 








The Harbor School 


6 to 8 


1997 


opened 


Lilia G. Frederick Pilot Middle School 


6 to 8 


2003 


opened 


Middle-High 








Josiah Quincy Upper School 


6 to I0 8 


1999 


opened 


High 








Fenway High School 


9 to 12 


1995 


converted 


Greater Egleston Community High School 


ungraded 


1996 


converted 


New Mission High School 


9 to 12 


1996 


opened 


Health Ca reers Academy 
(Horace Mann Charter) 


9 to 12 


1995 


opened 


Boston Arts Academy 


9 to 12 


1998 


opened 


Boston Day and Evening Academy 9 
(Horace Mann Charter) 


ungraded 


1995 


opened 


Boston Community Leadership Academy 


9 to 12 


2002 


converted 


TechBoston Academy 


9 to 10'° 


2002 


opened 


Another Course to College 


9, 1 1, 12" 


2003 


converted 




7 This school opened in 2004-05 with grades K0 and Kl, is adding one grade per year, and will eventually be 
a KO-5 school. Because this paper reports on school year 2003-04 data, this school is excluded from the data 
analysis. 

8 This school is adding one grade per year and is currently a 6-12 school. 

9 This school was Boston Evening Academy until September 2004, when it added a day program and was 
renamed Boston Day and Evening Academy. 

10 This school is adding one grade per year and is currently a 9-12 school. 

11 This school began as an 11-12 school, added one grade per year, and is currently a 9-12 school. 
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Pilot School Student 
Demographics 



T his report contains data from 2003-04 on three school types within the 

Boston Public School district: Pilot, BPS 12 , and Exam. The main purpose 

of this analysis is to understand how Pilot School student demographics and 

outcomes compare to those of students in BPS schools. 1 ’ Exam schools select 

When we went to your students based on prior academic achievement, while BPS and Pilot Schools do 

school for a field trip, I was not. Therefore, Exam school results are presented as a reference point but are 

. . . not discussed in depth, 

amazed at what I saw. I saw no 



one in the hallways wandering 
around. Everyone was in the 
classrooms learning and 
getting an education. That is 

the type of school I want to 

, ,99 

get into. 

— Pilot high school applicant 



Pilot School Students are Generally Representative of 
the District’s Students 

This section presents the demographics of Pilot Schools by five indicators to 
provide a portrait of the backgrounds and basic educational needs of the 
students in a school. These indicators also show whether proportionate 
numbers of students from each group are being represented across school 
types. As Pilot Schools are a subset of schools within a larger district, it is 
important to study enrollment patterns to ensure that Pilot Schools serve a 
population that is representative of the district. 



12 The BPS designation throughout this report includes all schools in the Boston Public Schools except for 
schools that have Pilot status and Exam schools. 

13 See Appendix A for methods. 
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High Schools 

Pilot high schools serve similar percentages of Black, White, 
Asian, and American Indian 14 students as compared to BPS high 
schools, excluding Exam schools. Pilots serve fewer Latino 
students than do BPS high schools. Pilot high schools serve 
similar percentages of students receiving free/reduced lunch and 
similar percentages of mainstream special education students as 
compared to BPS high schools. Pilot high schools serve fewer 
students who are not proficient in English. Pilot high schools 
serve fewer students who are classified as receiving substantially 
separate special education services. However, by the 2006-07 
school year, Pilot high schools will serve a representative 
percentage of these students. 



High School 
Race/Ethnicity 




American Asian Black White Latino 

Indian 



14 For simplicity, the race/ethnicity terminology used in this report is consistent with that 
used by the district and state, with the exception of students categorized as Hispanic, who 
will be referred to as Latino in this report. 



A M 

Pilot School 

Student 

Demographics 



High School Eligibility for 
Free/Reduced-Price Lunch 




High School, 
Other Demographics 




Not English Mainstream Substantially 
Proficient Special Ed Separate 



Special Ed 
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Middle Schools 





Pilot School 

Student 

Demographics 



Pilot middle schools serve a similar percentage of Black and 
White students as BPS middle schools. Pilot middle schools serve 
a higher percentage of Asian students and a lower percentage of 
Latino students than BPS middle schools. 1 ' 



Middle School Eligibility for 
Free/Reduced-Price Lunch 




Free/Reduced 
-Priced Lunch 



25 % 



20 % 



Middle School, 

Other Demographics 

■ BPS ■ Pilot ■ Exam 



Middle School 
Race/Ethnicity 




American Asian Black White Latino 

Indian 



15 % - 



12 




Proficient Special Ed Separate 
Special Ed 



15 The high number of Asian students can be attributed to one school, the Josiah Quincy 
Upper School, which is located in Boston’s Chinatown and has an enrollment that is 72% 
students of Asian descent. 
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Elementary Schools 

In comparison to BPS elementary schools, Pilot elementary 
schools serve similar percentages of Black and American Indian 
students. Pilot elementary schools serve a lower percentage of 
Asian and Latino students and 2.5 times the proportion of White 
students than BPS elementary schools. They serve a 55% greater 
proportion of mainstream special education students and a far- 
smaller proportion of students who are not English proficient or- 
ate in substantially separate special education. Pilot elementary 
schools serve proportionately 31% fewer students eligible for 
free/reduced-price lunch than BPS elementary schools. 

Further analysis of the elementary race and lunch-status 
demographic findings reveal that two of the five Pilot Schools 
serving the elementary level account for the higher percentage of 
White students and students who are not eligible for 
free/reduced-price lunch in Pilot elementary schools. One, the 
Lyndon School, has the highest percentage of White students of 
any elementary school in the district at 57%. Lyndon School and 
Mission Hill School rank second and fourth in the district for the 
lowest proportion of students eligible for free/reduced-price 
lunch at 39% and 45%, respectively. 
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Pilot School 

Student 

Demographics 



Pilot School Students are Becoming More 
Representative of BPS OverTime 



In the demographic analysis of the five indicators, on the whole Pilot high 
schools represent the overall student demographics of BPS. There are some 
differences in demographics at the middle level, and significant differences at 
the elementary level. In order to further investigate whether Pilots are moving 
closer to completely representative enrollment of the district population, the 
trends of Pilot School enrollment over three school years (2001-02, 2002-03, 
2003-04) were studied. 16 At every level, Pilot Schools are moving in the 
direction of serving a population of students more representative of those in 
the BPS. 



16 Data for each school year were analyzed as described in Appendix A: Methods, except that this analysis 
includes the two Pilot Schools that were newly opened in 2003-04. 
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B efore reporting on Pilot School student engagement and performance, 
this section describes how Pilot Schools tailor their educational programs 
to the needs of students. With the goal of providing high-quality education for 
every student, the Pilot Schools use their autonomy to design and implement 
organizational structures that support effective instruction and nurturing 
cultures. The indicators that are shown in the tables below have each been 
linked with a positive school climate and/or student outcomes (US Department 
of Education, 2000; Lee & Smith, 2001; Cotton, 1995; Hawley Miles & 
Darling-Hammond, 1998). Pilot Schools incorporate many of these practices, 
including smaller class sizes, 
longer instructional periods, 
longer school days, more time 
for teacher collaboration, 
and more time for teacher 
professional development. 

Due to each Pilot School’s 
autonomy, there is variation 
across the schools for each 
indicator. The tables below 
show the average value across 
schools in each school level. 



Without costing more, Pilot Schools have teachers who 
spend more of their time on professional development 
and collaboration and students who have more 
opportunities to develop caring relationships with adults. 
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Characteristics 
of Pilot Schools 



Elementary School Characteristics 

On average, Pilot elementary schools have smaller classes sizes than BPS 
elementary schools. Student and teacher days are longer, with students 
spending an average of just under one hour more and teachers spending just 
under two hours more in school each week. In contrast to BPS schools, Pilot 
School teachers appear to have significantly more time to collaborate with 
colleagues: almost three hours per week more than the minimum that is 
required for their counterparts in BPS schools. On average, Pilot elementary 
school teachers also participate in five more days of professional development 
per year. 



Elementary 

Schools 


Pilot 

Schools 


Boston Public 
Schools 


Average number of students seen by core academic 
teachers each day 


20 


24 


Length of student school day (minutes) 


370 


360 


Length of teacher school day, including 
after-school contracted faculty meeting time (minutes) 


409 


386 


Minutes per week of professional 
collaboration time 


216 


48 

(minimum) 


Number of full professional development days 


8 


3 



** 



find the smaller the classes 



Other data analysis show that the regular-education teachenstudent ratio is 
1:16, and several schools structure their staffing so that teachers and students 
stay together for more than one year. 



are, the easier it is for me to 
concentrate on what I’m doing 
to stay ahead of the game and 
keep focused. That’s the good 
thing about the school 
because it’s pretty small. 

— Boston Arts Academy stude nt 



Middle School Characteristics 

On average, Pilot middle schools have smaller class sizes than BPS middle 
schools and an average teachenstudent ratio of 1:17, and core academic 
teachers see an average of 52 students per day. Student and teacher days are 
longer, with students spending an average of two and a half hours more and 
teachers spending three and a half hours more in school each week. Pilot 
middle school teachers also have about four hours of professional 
collaboration time per week; BPS middle school teachers are not required 
under contract to have weekly professional collaboration time. On average, 
Pilot middle school teachers also participate in five more days of professional 
development per year. 
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Middle 

Schools 


Pilot 

Schools 


Boston Public 
Schools 


Average class size 


20 


28 


Length of student school day (minutes) 


400 


370 


Length of teacher school day, including 
after-school contracted faculty meeting time (minutes) 


438 


396 


Minutes per week of professional 


248 


no 


collaboration time 


minimum 


Number of full professional development days 


8 


3 



Characteristics 
of Pilot Schools 



In addition to having more time for teachers to collaborate and having 
longer student days, all Pilot middle schools have advisories, a structure that 
enables teachers and students to develop more personalized relationships. 
Students spend, on average, two hours per week in advisory. Many Pilot 
middle schools also offer long instructional periods, with all but one offering 
instructional periods longer than 60 minutes. 

High School Characteristics 

On average, Pilot high schools have smaller class sizes than BPS high schools 
and an average teachenstudent ratio of 1:14, and core academic teachers see 
an average of 64 students per day in core classes, advisory, and electives. 
Students spend an average of one hour more per week in school than their BPS 
counterparts. Teacher days are also longer, with teachers spending an average 
of three and two thirds hours more in school each week. On average, Pilot 
high school teachers have almost five hours of professional collaboration time 
per week, in contrast to teachers at BPS high schools, which are not required 
under contract to have weekly professional collaboration time. On average, 
Pilot high school teachers also participate in three more days of professional 
development per year. 



High 

Schools 


Pilot 

Schools 


Boston Public 
Schools 


Average 9th-grade English class size 


18 


28 

(all classes) 


Length of student school day (minutes) 


392 17 


380 


Length of teacher school day, including 
after-school contracted faculty meeting time (minutes) 


450 


406 


Minutes per week of professional 


285 


no 


collaboration time 


minimum 


Number of full professional development days 


6 


3 



17 This calculation excludes one outlier school. Boston Day and Evening Academy students attend school 
four days a week, for an average of 300 minutes per day because they are older students who have other 
responsibilities such as parenting and work. 
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In addition to having more time for teachers to collaborate and having 
longer student days, eight out of ten Pilot high schools implement advisories, a 
structure that enables teachers and students to develop more personalized 
relationships. In the schools with advisories, students spend an average of 108 
minutes per week in advisory. Many Pilot high schools also offer long 
instructional periods, with seven out of the ten schools offering instructional 
periods of 60 minutes or more. 

Summary 

Pilot Schools at all grade levels use their autonomies to organize themselves for 
high-quality instruction. Without costing more than BPS schools, Pilot Schools 
have smaller class sizes, lower student-teacher loads, and longer instructional 
periods. Teachers engage in more professional development and collaboration. 
Students have more opportunities to develop caring relationships with adults 
through advisories and smaller classes. While each school’s vision is different, 
all Pilot Schools have used their autonomies to create structures for greater 
personalization for students. 
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Pilot School 
Engagement 



Student 

and Performance 




Indicators of School Engagement 

The engagement and performance outcomes include a number of 
indicators, because no one indicator can accurately depict student 
or school success. This section contains outcomes in four areas: 
attendance, discipline, district leavers, and in-district transfers, 
each representing an aspect of student engagement. They are 
indicators that are collected at the district, state, and federal 
levels. High attendance rates, low discipline rates, and low 
mobility rates also indicate a positive, productive school climate. 

Pilot School outcomes in these areas are better than the BPS 
averages. With the exception of in-district transfers, differences 
between Pilot Schools and BPS schools become larger from the 
elementary to the high school level; Pilot high school averages 
for all four indicators are better than BPS averages. 

Attendance 

Attendance is an indicator of student engagement. High 
attendance rates are linked to positive student and school 
performance and are an indicator of school effectiveness (Crone 
et al, 1993; University of the State of New York, 1992). Student 
attendance is correlated with overall success in completing 
school. At the high school level, low attendance has been shown 
to be predictive of dropping out (Binkley & Hooper, 1989; Bryk 
& Thum, 1989; Sween, 1987). 



Median Attendance Rate 



■ BPS ■ Pilot ■ Exam 
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The differences in attendance rates between Pilots and BPS 
translate into Pilot students attending school, on average, almost 
two more days a year at the elementary level, just over a week 
more at the middle school level, and just over two weeks more at 
the high school level. 



25% 



20 % 



Out-of-School 
Suspension Rate 

■ BPS ■ Pilot ■ Exam 



15 % 



10 % 



5% 



0 % 




Elementary Middle High 



Pilot School students at each level have a median attendance 
rate that is higher than the BPS student rate, with the distance 
between the rates for the two types of schools increasing with 
each school level. While the median attendance rates are similar 
for BPS and Pilot elementary schools, Pilot School attendance 
rates are slightly higher at the middle school level and 
significantly higher at the high school level. Given that each 
percentage point in the attendance rate equals 1.8 days of 
school, 18 the differences in the attendance rates translate into 
Pilot School students attending school, on average, almost two 
more days a year at the elementary level, just over a week more 
at the middle school level, and just over two weeks more at the 
high school level. 

Out-of-School Suspensions 

The out-of-school suspension rate is an indicator of individual 
student engagement as well as of the culture of a school 
community. High-functioning school communities have lower 
suspensions rates. Fewer suspensions indicate fewer disruptions 
to learning in the school day and less instructional time lost 
(Cotton, 1990). Research suggests that out-of-school suspensions 
are not a desirable approach to discipline (Cotton, 1995; Pinnell, 
1985). A low rate may indicate better classroom management 
and a more integrated approach to dealing with inappropriate 
behavior in a school. 



18 Based on a 180-day school year. 
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As shown in this graph, Pilot elementary schools have a 
suspension rate that is one third that of BPS schools; Pilot middle 
schools have suspension rates that are slightly lower than BPS 
schools; and Pilot high schools have a rate that is about one half 
the rate of BPS schools. 

District Leavers and In-District Transfers 

While student background characteristics can be important 
determinants of a school’s mobility rate, up to half the variability 
of high school turnover rates can be attributed to school 
characteristics, including student:teacher ratio, quality of 
teachers, class size, and average daily attendance (Rumberger & 
Thomas, 2000). High student mobility has been “highly 
associated with a low level of student performance” at all levels 
(University of the State of New York, 1992). Therefore school 
characteristics influence mobility rate, and a high mobility rate is 
not a desirable outcome. 

The district leaver indicator includes students who transferred 
out of the district or dropped out of school. The in-district 
transfer rate includes all students who transferred from one 
school in the district to another for any reason other than 
completing the highest grade available at a given school. The 
district leaver and in-district transfer rates speak to the holding 
power of a school or the stability of the school population. 

Taken together, they serve as proxies for a school’s mobility rate. 

As shown in the graph, on average, Pilot Schools have a lower 
district leaver rate than the BPS. The distance between Pilot 
Schools and the district in the proportion of students who left 
school increased with each subsequent school level. At the 
elementary school level, the proportion of Pilot students who left 
their school is 27% lower than in BPS, while at the high school 
level it is 45 % lower. 

The distance between BPS and Pilot Schools for the in-district 
transfer rate is greatest at the elementary school level. The Pilot 
School in-district transfer rate is one third that of BPS. The Pilot 
middle school rate is 29% lower than the BPS rate, and the Pilot 
high school rate is 20% lower than the BPS rate. 
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Indicators of Student Performance 

The four indicators in this section — grade level retention, post- 
secondary education plans, post-secondary education 
participation, and MCAS scores — are measures of student 
performance. They provide a picture of how well schools are 
helping students to progress on grade level, to meet state 
standards, and to enroll in post-secondary education. As a 
whole, Pilot School students are performing better than the BPS 
average on all of the four performance indicators. As with the 
engagement indicators, the largest differences between the two 
types of schools are at the high school level. 




Two- and Four- Year 
College-Going Plans 




BPS Pilot Exam 



Grade Level Retention 

Grade level retention is not only a measure of the educational 
performance of students in a school, but it also predicts future 
academic achievement. There is a strong, positive correlation 
between grade level retentions and an increase in the risk of a 
student’s dropping out of school (Roderick, 1994; Goldschmidt 
& Wang, 1999). The grade level retentions that are reported in 
the table are the percentage of students in each school level and 
school type who were retained in grade from the 2002-03 to 
2003-04 school years. 

At the elementary level, Pilot Schools have a 33% lower grade 
level retention rate and at the middle level, Pilot Schools have a 
40% lower grade level retention rate than the BPS schools. 

As is the case with most of the engagement indicators, the 
difference between Pilot Schools and BPS schools in grade level 
retention is much greater at the high school level than at the 
elementary and middle school levels. Pilot high schools retained 
students at a rate that is less than half that of BPS high schools. 

Two- and Four-Year College-Going Plans 

The plans of high school graduates of both Pilot and BPS high 
schools are self-reported in district-administered exit surveys. 19 
They are an indication of what a student plans to do after 
graduation from high school and include options of two- and 
four-year colleges, other post-secondary education, work, 
military, and other. 

On average, 75% of graduates from each Pilot School 
planned to enroll in two- or four-year colleges, as compared with 
49% of BPS graduates from each school. 



19 Graduation rates are not reported in this study. However, graduation rates are an 
important indicator of school success. Ideally, the calculation of a school’s graduation rate 
should follow a cohort of students over their four years and be a measure of the proportion 
of students who graduate “on time” after four years. Data that affect the graduation rate 
include transfers into and out of a school, students who formally drop out, and those who 
drop out but are not recorded by the district. The data necessary to calculate cohort 
graduation rates have not been available to date. 
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Post-secondary Education Participation One Year after 
Graduation 

In today’s economy, a high school diploma is no longer sufficient to ensure 
future financial independence. Preparing students for the opportunity to attend 
post-secondary education is a major goal of all high schools. One measure of 
success for any high school is the number of its graduates who are in post- 
secondary education during the spring after graduation. The numbers reported 
below represent the percentages of graduates who were in post-secondary 
education — including four-year colleges, two-year colleges, and technical or 
vocational programs — one year after graduation. 20 

The graph shows the proportion of graduates in post-secondary education 
one year after graduation by school type and by Pilot School. The proportion 
of Pilot School graduates enrolled in post-secondary education one year after 
graduation is 18% higher than that for BPS graduates. Five out of six Pilot 
high schools are at or above the BPS average. The Pilot high school with the 
lowest proportion of college enrollees is Greater Egleston Community High 
School, a specialized school developed with a mission of serving older high 
school students and students who have dropped out of other schools. The 
school with the same average as BPS is Boston Community Leadership 
Academy, a new conversion Pilot School whose 2003 graduates experienced 
only the transition year to Pilot status. 
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Standardized Test Results 

The Massachusetts Comprehensive Assessment System (MCAS) exams are 
criterion-referenced tests administered by subject to students across the state. 
The MCAS is used as one measure of student performance. It is a high-stakes 
test at the tenth grade; students must pass the tenth-grade exams in order to 
graduate from high school. 

This study examines all tests administered for reading, English/Language 
Arts (ELA), and mathematics in 2004. It analyzes the results from each test 



20 Note that although this data was gathered in spring 2004, it is based on the class of 2003, which is the 
school year prior to the school year for most of the data in this report. 
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using two criteria: percent achieving advanced/proficient status and percent 
passing. Advanced/proficient equals the proportion of students at either of 
those achievement levels, and passing equals the proportion of students in the 
advanced, proficient, and needs improvement categories. The two categories 
are distinguished from each other because they convey different, important 
information — proficiency and pass rates. 

Pilot School students outperformed BPS school students at all grade levels 
in all tests. As a whole, the proportion of Pilot School students in the 
advanced/proficient categories ranged from 10 to 34 percentage points higher 
than BPS students. The proportion of Pilot School students passing each exam 
ranged from 3 to 27 percentage points higher than BPS students. 



Elementary School MCAS 

There are three elementary level tests: third-grade reading, fourth-grade ELA, 
and fourth-grade math. 

In third-grade reading, as a proportion, twice as many Pilot School students 
placed in the advanced/proficient category as BPS students. 21 The proportion 
of students in the passing category was 20% higher for Pilot Schools than for 
BPS schools. 

At the fourth-grade level, the proportion of Pilot School students passing 
the exam was slightly higher than for BPS students for both the ELA and math 
exams. The proportion of students scoring in the advanced/proficient 
categories was 59% higher in ELA and 76% higher in math. 



MCAS Grade 3 Reading 




■ BPS 

■ Pilot 



MCAS Grade 4 ELA 




Proficient 



MCAS Grade 4 Math 




Proficient 



21 Across the third grade, there were no students scoring in the advanced category. 
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Middle School MCAS 

The three middle school level MCAS exams are: sixth-grade math, seventh- 
grade ELA, and eighth-grade math. As at the elementary school level, the 
middle school level test results show that Pilot School students achieved a 
higher percentage than BPS students in both the advanced/proficient and 
passing categories. 

In the sixth-grade math test, Pilot School students scored at the 
advanced/proficient level at a rate more than twice that of BPS students. Pilot 
School students scored in the passing category at a rate that is 50% higher 
than BPS students. 

In the seventh-grade ELA exam, Pilot School students scored in the 
advanced/proficient and passing categories at a higher rate than BPS students. 
Seventh-grade Pilot School students performed in the advanced/proficient 
categories at a rate that is 33% higher than for BPS students. The Pilot School 
rate of passing was 9% higher than for BPS. 22 

At the eighth-grade level, the difference between Pilot School and BPS 
student performance is similar to that in the seventh grade. The overall 
performance of each group is lower at the eighth grade; fewer than two thirds 
of the students from each school type passed the exam. Pilot School students 
scored in the advanced/proficient category at a rate more than twice that for 
BPS students. Pilot School students also scored in the passing category at a rate 
26% higher than BPS students. 
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When we went to your 
school for a field trip, I was 
amazed at what I saw. I saw no 
one in the hallways wandering 
around. Everyone was in the 
classrooms learning and 
getting an education. That is 
the type of school I want to 
get into. 

— Pilot high school applicant 



MCAS Grade 6 Math MCAS Grade 7 ELA MCAS Grade 8 Math 




22 Exam schools serve grades 7-12 and enroll students based on exam scores and prior academic 
achievement. Their MCAS results are presented but not discussed further due to the exclusive nature of their 
admissions process. 
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MCAS Tenth-Grade ELA 
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High School MCAS 

At the high school level, MCAS is 
a high-stakes test. Students must 
pass both the tenth-grade ELA 
and math tests in order to 
graduate. If necessary, students 
are given more than one 
opportunity to pass each exam. 
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Tenth-grade analysis by student 

Pilot School students pass the 
tenth-grade ELA exam at a rate 
that is substantially higher than 
that for BPS students. While in 
2003-04 58% of BPS students 
passed the exam and qualified for 
a diploma, 84% of Pilot School 
students passed — a 45% 

difference. As a proportion, more than twice as many Pilot School students 
were in the advanced/proficient category as BPS students. 

Pilot School students scored in the advanced/proficient categories at a rate 
higher than BPS students for the tenth-grade MCAS math exam as well. 

Pilot School students passed at a rate that was 36% higher than BPS students 
and scored in the advanced/proficient category at a rate 38% higher than 
BPS students. 



MCAS Grade 10 ELA, MCAS Grade 10 ELA, 
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Tenth-grade analysis by school 

In addition to student-level analysis, the tenth-grade MCAS data was also 
analyzed at the school level. The results of the school-level analysis are 
presented in scatterplot graphs. 

The four scatterplot graphs show tenth-grade math and ELA exam results 
by school. 23 There are two graphs for each exam, one for advanced/proficient 
scoring levels and one for passing level. Each scatterplot shows two types of 
data: each dot represents the proportion of students in one school who were in 
either the advanced/proficient or passing category (depending on the graph), 
and each colored bar represents the average across all schools within that 
school type. For example, the tenth-grade MCAS ELA advanced/proficient 
graph shows that 40% to 50% of students at four Pilot high schools scored in 
the advanced/proficient categories. Across all Pilot Schools, an average of 33% 
of students at each Pilot School scored in the advanced/proficient category, 
which is almost twice the rate for BPS schools. In fact, the overall Pilot School 
average is greater than the averages of all but one BPS school in this category. 

On average, 80% of students at each Pilot School and 57% of students at 
each BPS school passed the tenth-grade ELA exam. All but one Pilot high 
school posted a passing rate greater than all but one of the BPS high schools. 

On average, 30% of students at each Pilot School and 23% of students at 
each BPS school scored in the advanced/proficient category for the tenth-grade 
math exam; the Pilot School rate is 30% higher than that for BPS schools. 

On average, 75% of students at each Pilot School and 58% of students at 
each BPS school passed the tenth-grade math exam. The Pilot School average 
pass rate was 29% higher than the BPS average pass rate. 



x\ 



Pilot Schools 
Engagement & 
Performance 



To graduate, you have to take a 
college course. It’s making it 
easier for you so when you 
leave high school and you 
really have to do full days of 
college courses, you’ve already 
been in a college class so you 
kind of have an outlook on 
what’s going to go on in class, 
what kind of work you’re going 
to be doing. You’re just getting 
a feel of college life before 
you’re even there. 

— Greater Egleston Community 
High School student 
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23 Another Course to College and Boston Adult Academy are excluded from the school-level analysis because 
both had fewer than 20 students taking each test. 
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Conclusions 



T he findings in this report show that Boston Pilot School students are 
faring well on a wide range of indicators of engagement and 
performance. The data confirm previous reports and demonstrate consistently 
positive findings for Pilot School students for the years 1997-2004 (CCE,250 
2001a; CCE, 2004a). Taken together, the Pilot School results strongly suggest 
that personalized, autonomous schools are able to create nurturing learning 
environments in which students achieve academically. 

Families have chosen Pilot Schools in increasing numbers, indicating a 
demand for safe, caring, academically excellent schools. The Pilot 

Schools Network has expanded in its decade of existence from 5 schools in 
1995 to its current 19 schools, serving approximately 10% of the BPS student 
population. The demand for these small, autonomous schools has increased, as 
demonstrated by full enrollments and the number of traditional BPS schools 
interested in learning more about becoming Pilot Schools. The wide variety of 
Pilot Schools — distinguished by mission, structure, instructional approach, 
and target population — provides Boston families with increased choices. 

Grade configurations range from the early childhood, elementary, middle, and 
high schools to those spanning two levels (such as K-8 and 6-12 schools). 
Curricula offered span a range of foci, including social justice, technology, 
health professions, and the arts. When families are offered a choice of schools 
from which to match personal interests and strengths, they are better able to 
find schools in which their students will be highly engaged and can achieve 
academically. 
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The Pilot School results strongly suggest that personalized, 
autonomous schools are able to create nurturing learning 
environments in which students achieve academically. 




Pilot Schools do not select students based on prior academic 
achievement or testing. This policy is in contrast to Exam schools, which 
are selective by entrance exam results and prior academic performance. At the 
high school level, Pilot Schools serve students representative of the district in 
every category except students not proficient in English and students receiving 
substantially separate special education services. However, Pilot Schools are 
serving an increasingly representative percentage of students in this latter 
category. 

At the elementary and middle school levels, Pilot Schools are part of the 
district’s controlled-choice student assignment system. Eight of the ten schools 
serving elementary and middle grades follow the district assignment process by 
zone, and the other two are K-8 citywide schools, drawing from residents 
across Boston. At both the elementary and middle school levels, Pilot Schools 
closely represent the district in the proportion of both students not proficient 
in English and students in mainstream special education. As with the high 
schools, elementary and middle Pilot Schools have a goal of equitable 
enrollment by 2006 for students receiving substantially separate special 
education services. 

One challenge that has concerned the Network is that Pilot Schools have 
historically served fewer students of color and low-income students at the 
elementary level (CCE, 2001a; CCE, 2004a). As the Network expands and 
matures, Pilot Schools are moving toward representing similar proportions of 
students by every indicator. 

On indicators of student engagement, Pilot School student outcomes 
are notable. This report has focused on four indicators of student 
engagement, all of which are predictive of academic achievement. The 
promising outcomes in Pilot Schools include: 

• The median attendance rate was higher in Pilot Schools at all levels. 

• The out-of-school suspension rate was lower in Pilot Schools at all levels. 
In particular, Pilot Schools showed proportionately one third of the out- 
of-school suspensions of BPS schools at the elementary level and half the 
suspensions at the high school level. 

• The district leaver rate was substantially lower in Pilot Schools at all 
levels. 

• The in-district transfer rate was lower in Pilot Schools at all levels. 
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Pilot School students are performing well academically and better 
than the BPS average. This report focused on four indicators of student 
performance: grade level retention, college-going plans, college enrollment, 
and MCAS scores. On every indicator, Pilot Schools outpaced the BPS average. 

• Grade level retentions in Pilot Schools were lower than in BPS schools at 
every level. BPS high schools retained students in grade at over twice the 
rate of Pilot high schools. Given the correlation between grade level 
retention and high school dropout rates, this is an important difference. 

• Pilot high school graduates enroll in college at a rate that is 18% higher 
than BPS high school graduates. 

• Pilot Schools surpassed BPS performance on every MCAS exam at every 
grade level tested. 

• As a proportion, significantly more Pilot School students scored in the 
advanced/proficient categories than BPS students for every test. 

Pilot Schools use their autonomy to create conditions for high-quality 
teaching and learning. Previously, CCE has documented evidence of 
effective Pilot School curriculum and assessment approaches, for example, 
multiple performance-based assessments that help to determine a student’s 
promotion and graduation status (CCE, 2004b). The Pilot Schools Network is 
explicit about the central role of improving teaching and learning in creating 
and sustaining successful schools. 24 The data on school characteristics and 
student outcomes in this report demonstrate that Pilot Schools use their 
autonomy to create longer blocks of learning time, low student-teacher ratios, 
and other structures that support high expectations and achievement. They 
have created a more personalized learning environment through their 
scheduling and staffing autonomy. Pilot Schools emphasize time for faculty 
collaboration and planning — a critical building block to improving and 
maintaining a school’s culture and performance. 



24 See Appendix D for Network vision and principles. 
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Implications for Other Schools 
and Districts 



T he Boston Pilot Schools are the product of an innovative partnership 

between the Boston Public Schools and the Boston Teachers Union. Pilot 
Schools are the only network of in-district schools in the nation which, by 
virtue of a teachers union contract, have charter-like autonomies. While there 
are many singleton high performing schools in public school districts, they 
lack the strength that comes from the cross-school collaboration, professional 
development, and advocacy that Pilot Schools have. The Pilot Schools 
Network focuses simultaneously on the needs of each school and on 
developing the external supports necessary for all to succeed. It keeps teaching 
and learning at the center while advocating for systemic change. The following 
implications are based on the success of Pilot Schools documented in this 
report. 

Pilot Schools serve their students well because they have autonomy 
along with increased accountability. The Pilot autonomies enable schools 
to use resources flexibly for increased personalization and improved outcomes. 
Pilot Schools’ structure, culture, and instruction are distinct from those of 
many district schools because autonomy allows schools to organize in a 
fundamentally different way. The school characteristics data presented in this 
report show that Pilot Schools are using their budget, staffing, and scheduling 
autonomy creatively and flexibly. 

• Pilot Schools structure themselves for increased personalization among 
students, among teachers, and between teachers and students. Structural 
innovations include small class sizes, low student:teacher ratios, longer 
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Implications for 
Other Schools 
& Districts 



instructional periods, looping or multiage groupings in classrooms, and 
student advisories. 

• While Pilot Schools are diverse in mission and culture, they share a 
commitment to collaboration and inquiry at all levels. Staff have more 
professional development time, more time to reflect together on student 
work and teacher practice, and greater decision-making authority over 
the content of their professional collaborative time. 

• Through their curriculum and assessment autonomy, Pilot Schools’ 
instructional approaches are designed to meet student needs. Pilot School 
students spend more time in school and in core academic classes. There 
are increased professional development opportunities for teachers and 
longer instructional blocks. Pilot Schools are anchored by performance- 
based assessments for promotion and graduation, which raise the stakes 
for students, deprivatize curriculum and instruction for teachers, and 
involve community members to give the assessments increased credibility 
and a sense of purpose (CCE, 2004b). 



The Pilot Schools Network focuses on the needs of each 
school, and on developing the external supports necessary 
for all to succeed. It keeps teaching and learning at the 
center while advocating for systemic change. 



Pilot autonomy comes with accompanying high expectations. Pilot Schools 
undertake a five-year-cycle of school quality reviews, using a set of 
benchmarks that articulate the criteria for high-performing schools. Along 
with student outcomes, each school is assessed for vision, leadership and 

governance, teaching and learning, 
professional development, and 
family and community 
engagement. 

While autonomy is 
necessary, school faculties must 
have the vision and ability to 
implement the autonomies for 
improving the school’s 
structure, culture, and 
instruction. 



Autonomy and small size are critical for creating a high-performing 
school. Research suggests that schools able to create environments in which 
students are well known to their teachers and in which teachers have adequate 
time to collaborate are more successful in meeting their students’ needs 
(Hawley Miles and Darling-Hammond, 1998; Newmann, 1996). Pilot Schools 
are better able to create these personalized and collaborative cultures through 
use of their autonomy and small size — all but two Pilot Schools have 500 
students or fewer, and the two larger schools are broken down into smaller 
academies. 

Research shows that, academically, students in small schools do as well as, 
and often better than, those in large schools (Cotton, 1995). Student attitudes 
are more positive, engagement is higher, and access to learning opportunities is 
greater. Small schools are also more cost-effective, based on cost-per-graduate 
(Stiefel et al, 2000). 
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While autonomy and small size are critical elements of the Pilot model, the 
two newest Pilot Schools opened with a different set of conditions. Each 
school opened with full enrollments of 700, a size that far exceeded 
enrollments in all other Pilot Schools. In addition, due to opening in a year 
with double-digit budget cuts, each school had over 25% of their staff 
involuntarily assigned to the school in the first year, severely limiting their 
staffing autonomy. The schools did not have control over some aspects of 
student assignment and programming. As a result, both schools had 
understandably low engagement and performance outcomes in their first 

25 

year. 

Future new start-up Pilot Schools approved by the district and teachers 
union should be small, should roll out grade by grade, and should be afforded 
the full set of autonomies in the start-up year. 

The Pilot Schools’ positive outcomes suggest the schools’ practices 
and strategies should be shared with the district and other educators. 

Pilot Schools were created to serve as laboratory sites for the district, in which 
flexibility allows for innovations that would in turn inform other schools. The 
data in this paper show that Pilot Schools have fewer students leaving through 
transfer or dropping out and lower in-district transfer rates, suggesting that 
Pilot School students have found good matches in their schools. Pilot Schools’ 
diversity in mission and structure allows a range of students to find schools 
that are a good fit. As the district looks for strategies to create schools that are 
more responsive to student needs, Pilot Schools should serve as a model for 
increasing the choice options within the district. 

Another Pilot School indicator that correlates with higher academic 
achievement is their lower grade level retention rate. Almost all Pilot Schools 
promote students based on competency rather than courses or credits accrued. 
Teachers use multiple and varied assessments, including performance 
assessments, to determine student promotion. A detailed study of how Pilot 
Schools develop and implement a system of competencies and assessments may 
inform other educators about strategies for improving promotion and 
graduation rates. 

The Pilot Schools should continue to document their innovative practices, 
and the Pilot Schools Network and district should collaborate on developing 
more ways of sharing these practices with other schools and districts. 

The district’s and teachers union’s support of Pilot Schools is crucial 
to their success. Having a network of autonomous schools changes the 
relationship between those schools and the district. While districts 
traditionally mandate reforms and monitor their implementation, the BPS 
central office’s role has shifted to one of providing services and supporting 
Pilot Schools in the use of autonomies. Districts that create Pilot Schools must 
also develop strong accountability measures for autonomous schools, prepare 
leaders in the implementation of autonomy, and provide discretionary services 
needed by schools. 




25 See Appendix C for individual school outcomes. 
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Teachers are at the center of schools and should be included in all phases of 
designing and implementing Pilot Schools. When Pilot Schools are being 
created by a district, teachers union co-planning results in greater “buy-in” for 
the new schools. The union can support improvement of Pilot Schools and the 
Pilot School model through participation in the school quality review process 
and sharing of practices between Pilot School and BPS teachers. 

A collaborative effort between the district and the teachers union includes 
new and expanded roles for each entity and ensures a stronger, more 
responsive Pilot Schools Network. 

Success is achievable in the public school system. The Pilot Schools 
Network has grown in size in its decade of existence. At a Boston Foundation 
planning grant session in 2002, one quarter of BPS schools expressed interest 
in how to become Pilot Schools. Students cite academic rigor, interesting 
curricular offerings, and caring, nurturing teachers as the features that attract 
them to Pilot Schools (Doyle et al, 2003). Meanwhile, 26% of Boston school- 
age residents now choose to attend school outside the district. 26 Boston needs 
to continue to expand choice options for its families so that they will stay in 
district schools or return if they have left. 

As members of the public school system, and with the same per pupil 
expenditure, Pilot Schools have outpaced the BPS average on every measure of 
engagement and achievement. In coming years, we expect the Boston Pilot 
Schools Network to deepen and expand its work in Boston by serving greater 
numbers of students. As a laboratory of innovation and improvement, the 
Network is also in a key position to assist other districts in creating successful 
small and autonomous schools. Through their choice, commitment, 
educational excellence, and innovation within the BPS system, Pilot Schools 
have made the system stronger and more successful. 



26 BPS website, http://www.boston.kl2.ma.us/bps/bpsglance.asp 
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Appendix A: Methods 

Data Sources 

The data used in this report come from several sources. The Boston Public 
Schools provided CCE with a database of indicators in 52 categories for the 
end of the school year 2003 -04 27 for every student who was enrolled at any 
point during the school year. Data was received at the individual student level, 
with randomly generated unique student identifiers to protect student 
confidentiality. The demographic and engagement data was also reported to 
the Massachusetts Department of Education through its Student Information 
Management System (SIMS). 2S Data points for each student include 
race/ethnicity, lunch status, date of birth, grade level, special education 
placement, English proficiency, days of attendance, and number of 
suspensions. In addition to the SIMS indicators, the database includes results 
of the Massachusetts Comprehensive Assessment System (MCAS) exams in 
English Language Arts and/or math for students in grades 3, 4, 6, 7, 8, and 10. 

A second source of data was the Boston Private Industry Council (PIC), 
who provided data on post-secondary participation of Boston high school 
graduates. 

Data were also collected directly from schools on a number of quantitative 
indicators of school practice. (These second two data sources are described in 
more detail below.) 

A fourth source of data was the Massachusetts Department of Education 
website, for data on college plans. 

School Types 

All schools included in the analysis are part of the Boston Public Schools 
(BPS). All schools within BPS may be classified as one of three types: 
traditional district schools, Pilot Schools, and Exam schools. 29 Each of the 
three types of schools is defined below. 

Boston Public Schools — Includes all schools in the Boston Public 
Schools except for schools that have Pilot status (Pilot Schools) and those that 
require students to take and pass an entrance examination (Exam schools). 
Therefore, the 2003-04 BPS sample analyzed in this report includes 107 
schools and 85% of all students in the district. For simplicity, these schools are 
referred to as “BPS” schools in this report. 

Pilot Schools — Includes all schools that have been granted Pilot status by 
the district, except for the one Early Learning Center, the two schools that 
were newly opened in 2003-04, and the one school that opened in 2004-05. 
Orchard Gardens and Lilia G. Frederick, the two schools that were newly 
opened in 2003-04, were not included in the analysis in the body of the report 



27 BPS provided the data for all schools except Boston Day and Evening Academy and Health Careers 
Academy. In addition to being Pilot Schools, both schools are also Horace Mann Charter Schools. The data 
from each of these two schools came directly from the schools in the SIMS format. 

28 For more information on the Massachusetts Department of Education’s SIMS, including indicators and 
their codes, see http://www.doe.mass.edu/infoservices/data/sims/. 

29 Schools that serve as Early Learning/Early Education Centers and those that exclusively serve students 
with severe special learning or behavioral needs were not included in the analysis (see Appendix E for list). 
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because they opened without the opportunity to exercise the full autonomy 
that all other Pilot Schools have. Therefore, the 2003-04 Pilot School sample 
includes 15 schools, serving 7% of students in the district. 

Exam Schools — Includes three schools that base admission on entrance 
examination scores and prior academic achievement. Each school serves 
students in grades 7-12. In 2003-04, there were three Exam schools serving 
9% of all students in the district. 

School Levels 

All analyses, with the exception of the MCAS results, which are by test grade 
level, are divided into three school levels: 

Elementary — Kindergarten through grade 5; 

Middle — Grades 6 through 8; and 
High — Grades 9 through 12. 

Several schools span these traditional grade boundaries. 30 For the schools 
that span traditional school grade levels, student data were analyzed by 
appropriate levels based on grade. For example, for all K-8 schools, students 
in grades K-5 were included in the analysis of elementary schools while 
students in grades 6-8 were included in the analysis of middle schools. 

The following table shows the numbers of schools at each school level by 
school type. ’ 1 





Elementary 


Middle 


High 


BPS 


72 


24 


18 


Pilot 


5 


7 


10 


Exam 


N/A 


3 


3 



Analyses of the SIMS and MCAS data were completed using SPSS software. 
All data is aggregated by school level and school type. To facilitate data 
analysis in SPSS according to school level and school type, two variables were 
created for each student. The first is a school level variable that identifies each 
student as an elementary, middle, or high school student based on his/her 
grade level. The second is a school type variable that identifies each student as 
either a BPS, Pilot, or Exam school student. Both variables were used in all 
demographic and engagement analyses, while the school type variable was 
used in the MCAS analyses. 



30 Schools spanning grade levels were: three K-8 Pilot Schools, one 6-10 Pilot School, six K-8 BPS schools, 
and three 7-12 Exam schools. 

31 Schools that span grade levels were included in the corresponding totals. For example, schools that serve 
students in grades Kindergarten to 8 are included in both the elementary and middle totals. 
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Demographics 

Four major demographic indicators were calculated for each school 
level/school type: Race/ethnicity, eligibility for free or reduced-price lunch, 
special education status, and English proficiency. 32 Each analysis represents the 
proportion of students in each particular school level and school type by 
indicator.” For example, the proportion of Black students in Pilot middle 
schools is calculated as the total number of Black students in Pilot middle 
schools divided by the total number of students in Pilot middle schools. 

Race/Ethnicity — In Massachusetts, students are classified into one of five 
categories: American Indian/Alaska Native; Asian/Pacific Islander; Black; 
White; or Hispanic. The proportions were calculated for each category. 

Free/Reduced-Price Lunch Eligibility — Eligibility status for free or 
reduced-price lunch is the most commonly used indicator of the income level 
of students in a school. In the absence of complete and accurate data on 
household income, lunch status serves as a proxy for income. Eligibility was 
indicated in one of three categories: the student is not eligible for either; the 
student is eligible for free lunch; or the student is eligible for reduced-price 
lunch. The second two categories were collapsed into one — eligible for 
free/reduced-price lunch. 

Special Education Status — Students are classified into 13 categories in 
the Massachusetts Department of Education’s Special Education Placement 
variable. Of those, 7 categories applied to students in the whole district in 
2003-04. To reflect predominant practices and simplify reporting, a new 
variable was created that combined the classifications into three groups: not in 
special education, in mainstream special education, and in substantially 
separate special education. 34 The classifications for both mainstream (502.1, 
502.2, and 502.3) and substantially separate (502.4) special education 
categories are consistent with students’ federal special education prototypes. 

Limited English Proficiency (English Language Learners) — Students 
are reported in two categories: those who are capable of performing ordinary 
classwork in English and those who are not. 



32 Students who were enrolled in school at the end of the 2002-03 school year but attended zero days of 
school in 2003-04 were excluded from all analyses except for the district leaver and grade level retention 
analyses. 

33 All values in this report are rounded to the nearest whole number. 

34 The following three special education placement categories were collapsed into “not in special education”: 
“not a Special Education student”; “not a Special Education student, but was previously a Special Education 
student during the current school year”; and “3-5 year olds, General Education serving as role models in Pre- 
K classes.” The following were collapsed into “mainstream special education”: “All ages, full inclusion” and 
“All ages, partial inclusion.” The following category was recoded as “substantially separate special 
education”: “All ages, substantially separate classroom.” The only special education category that applied to 
district students in 2003-04 that is not included in this analysis is the “All Ages Public Separate Day School”: 
This category refers to students who attend schools that were excluded from the overall analysis. The other six 
categories did not apply to students in the district in 2003-04 and therefore were not part of the recoded 
variable. 
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Engagement 

Attendance — Attendance was reported as the number of days that a student 
attended school. A second variable, membership, indicates the number of days 
that a student was enrolled in the school. An attendance rate variable was 
calculated for each student by dividing the number of days attended by the 
number of days of membership. The case summary function was then used to 
determine the range and median for each school level/school type. The figure 
reported is the median attendance rate for students in a school level/school 
type. 

Discipline — This indicator reports the proportion of students who have 
been suspended (received out-of-school suspensions) at least once during the 
school year. It does not take into account the number of times that an 
individual student was suspended from school. 

District Leavers — District leavers include students who transferred out of 
the district, dropped out, were expelled, or were enrolled at the end of the 
previous school year but did not return for the 2003-04 school year. 

In-District Transfers — This indicator includes students who changed 
schools within Boston Public Schools between the end of the 2002-03 school 
year and the end of the 2003-04 school year. 1 ' Transferring students were 
associated with two schools; rates were determined according to the school 
level and type that they left. Three data points were utilized in the analysis: 
June 2003, October 2003, and June 2004. 

An October 2003 -June 2004 transfer rate was calculated for all students 
who were new to the district or new to kindergarten in 2003-04. A June 
2003 -June 2004 rate was calculated for all students who were in the district 
in 2002-03, except for those students who were in either fifth or eighth grade. 
Due to the fact that not all of the Boston elementary and middle schools end at 
grades 5 and 8, respectively, transfers of students in these grades were 
calculated separately. Finally, the results of each analysis were aggregated to 
arrive at the in-district transfer rate. 36 

Performance 

Grade Level Retention — This measure is the proportion of students in each 
school level/school type who were in the same grade in 2003-04 as in 
2002 -03. 37 Grade level retentions were calculated by subtracting a student’s 
grade level in 2002-03 from his/her grade level in 2003-04. Boston Evening 
Academy and Greater Egleston Community High School were not included in 
this analysis because they are competency-based schools that do not use 
traditional grade levels. 



35 Health Careers Academy and Boston Day and Evening Academy, which are Horace Mann Charter Schools 
in addition to being Pilot Schools, were not included in this analysis because comparable data were not 
available for this indicator. 

36 For more detail on in-district transfer rate methods, please contact the authors. 

37 Grade level retentions also include a small number of students who were in a lower grade level in 
2003-04 than they had been in 2002-03. There are times when students arrive in the district without 
educational records and are subsequently placed in an inappropriate grade level. 
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College Plans — Data represent twelfth-grade students’ college-going 
plans in May 2004, as reported to the district. Data were downloaded from 
the Massachusetts Department of Education website’ 8 into Microsoft Excel. 
Data were not available for two BPS high schools and one Pilot high school. 
Data were obtained directly from one other Pilot high school. The rate 
reported for each school type is the average of each school’s proportion of 
students planning to go to either two- or four-year colleges. 

Participation in Post-secondary Education — Survey data on post- 
secondary participation of 2003 high school graduates were provided by the 
Boston Private Industry Council (PIC), an organization that provides school- 
to-work services to Boston high schools. The overall survey response rate was 
81% of 2003 Boston high school graduates; the data provided by PIC are 
derived population estimates. 39 The rate presented in this report is the 
proportion of graduates who were enrolled and attending a four-year college, 
two-year college, or technical or trade school one year after graduation. 
Results are displayed as the proportion of each Pilot high school as well as the 
average proportion of interviewees in each school type who were enrolled in 
post-secondary education one year after graduation. 

MCAS — Student performance on the Massachusetts Comprehensive 
Assessment System (MCAS) exams was used as an indicator of students’ 
academic performance. Students take MCAS tests in grades 3-10. This report 
includes an analysis of all of the reading, English/Language Arts, and math 
tests administered in grades 3, 4, 6, 7, 8, and 10. MCAS performance data 
were provided at the categorical (ordinal) level. For each student who 
participated in the MCAS during the spring of that school year, results were 
reported at four categorical achievement levels: Advanced, Proficient, Needs 
Improvement, and Warning/Failing. 40 Results are presented in two ways: the 
percentage of students in the Advanced and Proficient categories and the 
percentage of students passing the test, with passing encompassing the 
categories of Advanced, Proficient, and Needs Improvement. Results are 
presented as the percentage of students in each of the two collapsed categories 
by school type. 



38 Source of college plans data: http://profiles.doe.mass. edu/plansofhsgrads.aspx?mode=school&corderBy=. 

39 Estimates are based on weighted sample results. For more information, see Ishwar Khatiwada and Andrew 
Sum’s College Enrollment and Labor Market Outcomes for Class of 2003 Boston Public High School 
Graduates: Key Findings of the Winter/Spring 2004 Follow-up Surveys published in December 2004 by the 
Boston Private Industry Council. Available at http://www.bostonpic.org/about/Classof2003- 
CollegeEnrollandLaborMarketOuts.pdf. 

40 The fourth category is called “Warning” for students in grades 3-8 and “Failing” for students in grade 10, 
signifying the high-stakes nature of the tenth-grade exams. 
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Appendix B: Methods for School 
Characteristics 

Pilot School Characteristics 

Pilot School characteristics are quantitative indicators of staffing, scheduling, 
and curriculum that affect a school’s structure, culture, and instruction (CCE, 
2001b). While the rest of the data in the report are from the 2003-04 school 
year, these data are from the 2004-05 school year. Raw data were collected 
and school practices were calculated for each Pilot School. The results were 
then averaged within each school level and are presented as such. Where 
available, Boston district comparisons are also presented. Corresponding BPS 
figures were obtained from the Boston Public Schools FY 2005 Budget and 
School Hours website (www.boston.kl2.ma.us) and the 2003-06 Boston 
Teachers Union contract. 



Report Review Process 

This study was completed by research staff at the Center for Collaborative 
Education, the Pilot Schools Network’s convening organization. The report 
was reviewed by the CCE Research Advisory Group, other researchers, and 
key Boston Public Schools staff. 
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Indicator 



Pilot Data Source 



Pilot Method 



BPS Data Source 



Average number of students 
seen by core academic 
teachers each day in core 
academic subjects 


Teacher schedules and 
course lists 


Add number of students seen in core 
classes each day of the week and 
then divide by 5. 


Not available 


Average class size 


Course list or class rosters, 
depending on the school 


Average the enrollment of regular 
education classes. 


BPS FY2005 budget and BTU 
2003-06 contract 


Average 9th-grade English 
class size 


Course list or class rosters, 
depending on the school 


Average the enrollment of all regular 
education 9th-grade English classes. 


BPS FY2005 budget and BTU 
2003-06 contract 


Regular education 
teacher:student ratio 


Staff list and total enrollment 
by education type 


Divide all regular education students 
by the total number of regular 
education teachers, including core 
teachers and specialists. 


Not available 


Length of student school day 
(minutes) 


School schedule 


Add minutes that students are 
required to be in school each day of 
the week and divide by 5. 


BPS school hours webpage 
(predominant hours for each 
grade level) 


Length of teacher school day, 
including after-school 
contracted faculty meeting 
time (minutes) 


Work Election Agreement, or if 
noWEA available, school 
master schedule or staff 
handbook 


Add minutes that teachers are 
required to be in school each day of 
the week and divide by 5. 


BPS school hours webpage, 
BTU 2003-06 contract 


Minutes per week of 
professional collaboration time 


Teacher schedules and Work 
Election Agreement 


Add minutes per week that each 
teacher has for professional 
collaboration, regularly scheduled 
staff meetings, and professional 
development.Add all teachers’ time 
together and divide by the number 
of teachers. 


BTU 2003-06 contract 


Number of years students and 
teachers stay together 


School personnel 


Give actual range. 


Not available 


Advisories 


Teacher and student schedules 


Indicate yes or no. 


Not available 


Minutes/week in advisories 


Teacher and student schedules 


For each individual student, add all 
time spent in advisories; then average 
all students. 


Not available 


Number of full professional 
development days 


Work Election Agreement and 
School Personnel 


Add actual number of required full- 
day professional development 
sessions. 


BTU 2003-06 contract 
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Appendix C: 2003-04 Pilot School 
Outcomes by School 



The following tables show the rates for each outcome indicator for each 
individual Pilot School and the BPS average for the school level. All numbers 
are percentages. Orchard Gardens K-8 School and Lilia G. Frederick Pilot 
Middle School are included in these tables. 
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Pilot School Student Engagement, by School 





Attendance 

Rate 


Suspension 

Rate 


District 
Leaver Rate 


In-District 

Transfer 

Rate 


Elementary Schools 










Lyndon 


97 


2 


5 


4 


Mason 


99 


1 


7 


4 


Mission Hill 


97 


0 


13 


2 


Orchard Gardens 


96 


4 


5 


2 


Young Achievers 


96 


0 


9 


4 


BPS Elementary Schools 


96 


3 


1 1 


12 


Middle Schools 










Harbor 


97 


26 


16 


8 


Lilia G. Frederick 


93 


27 


12 


2 


Lyndon 


96 


10 


9 


10 


Mission Hill 


97 


0 


15 


8 


Orchard Gardens 


97 


16 


9 


2 


Quincy Upper 


99 


6 


3 


15 


Young Achievers 


96 


0 


7 


4 


BPS Middle Schools 


94 


14 


14 


14 


High Schools 










Another Course to College 


93 


2 


15 


4 


Boston Arts Academy 


95 


0* 


6 


3 


Boston Community Leadership Academy 


94 


18 


14 


6 


Boston Day and Evening Academy 


92 


0 


20 


Not available 


Fenway 


95 


0 


6 


2 


Greater Egleston Community High 


75 


0 


27 


5 


Health Ca reers Academy 


95 


9 


3 


Not available 


New Mission 


99 


0 


12 


3 


Quincy Upper 


97 


7 


3 


10 


TechBoston Academy 


97 


0 


1 1 


3 


BPS High Schools 


89 


9 


22 


5 



* Boston Arts Academy’s suspension rate is actually 0.5%. The other schools listed as zero were actually zero. 
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Pilot School Student Performance, by School 



Percent of 
students 
retained in 
grade 


Percent of 
2004 

graduates 
planning to 
attend 2- or 
4-year 
colleges 


Proportion 
of 2003 
graduates 
in post- 
secondary 
education 
one year 
later 


Elementary Schools 




Not applicable 


Lyndon 


3 


Mason 


3 


Mission Hill 


4 


Orchard Gardens 


12 


Young Achievers 


7 


BPS Elementary Schools 


6 


Middle Schools 




Harbor 


6 


Lilia G. Frederick 


8 


Lyndon 


1 


Mission Hill 


4 


Orchard Gardens 


7 


Quincy Upper 


2 


Young Achievers 


0 


BPS Middle Schools 


5 


High Schools 








Another Course to College 


6 


65 


Not applicable 


Boston Arts Academy 


8 


78 


86 


Boston Community Leadership Academy 


12 


66 


67 


Boston Day and Evening Academy 




59 


Not available 


Fenway 


4 


70 


94 


Greater Egleston Community High 


Not available 


Not available 


50 


Health Careers Academy 


2 


90 


92 


New Mission 


1 1 


100 


71 


Quincy Upper 


2 


Not applicable 


Not applicable 


TechBoston Academy 


6 


Not applicable 


Not applicable 


BPS High Schools 


19 


49 


67 



44 Progress and Promise: Results from the Boston Pilot Schools 



www.cce.org 




Pilot School MCAS Results, by School 



Elementary Grades 


Grade 3 MCAS Reading 


Grade 4 MCAS ELA 


Grade 4 MCAS Math 




% advanced/ 
proficient 


% passing 


% advanced/ 
proficient 


% passing 


% advanced/ 
proficient 


% passing 


Lyndon 


80 


96 


64 


89 


49 


85 


Mason 


67 


96 


55 


94 


58 


94 


Mission Hill 


60 


80 


24 


43 


14 


29 


Orchard Gardens 


12 


60 


3 


47 


0 


15 


Young Achievers 


54 


98 


24 


65 


12 


53 


BPS Elementary Schools 


34 


80 


29 


74 


21 


68 



Middle Grades 


Grade 6 Math 


Grade 7 ELA 


Grade 8 Math 




% advanced/ 
proficient 


% passing 


% advanced/ 
proficient 


% passing 


% advanced/ 
proficient 


% passing 


Harbor 


9 


47 


31 


84 


5 


39 


Lilia G. Frederick 


2 


24 


32 


69 


4 


28 


Lyndon 


49 


71 


82 


91 


50 


63 


Mission Hill 


7 


40 


23 


46 


24 


29 


Orchard Gardens 


4 


25 


21 


63 


3 


18 


Quincy Upper 


54 


81 


55 


93 


36 


70 


Young Achievers 


38 


88 


32 


95 


33 


56 


BPS Middle Schools 


17 


46 


36 


80 


12 


43 



High Schools 


Grade 10 ELA 


Grade 1 0 Math 




% advanced/ 
proficient 


% passing 


% advanced/ 
proficient 


% passing 


Another Course College 


Not applicable 


Not applicable 


Not applicable 


Not applicable 


Boston Arts Academy 


46 


95 


33 


88 


Boston Community Leadership Academy 


24 


73 


30 


77 


Boston Day and Evening Academy 


Not available 


Not available 


Not available 


Not available 


Fenway 


35 


85 


33 


88 


Greater Egleston Community High 


5 


30 


0 


25 


Health Ca reers Academy 


43 


95 


28 


78 


New Mission 


24 


81 


24 


75 


Quincy Upper 


40 


87 


43 


77 


TechBoston Academy 


48 


93 


50 


95 


BPS High Schools 


17 


58 


24 


59 
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Appendix D: Boston Pilot 
Schools/Horace Mann Network Vision 
Statement; and Principles and Practices 

Vision Statement 

The Boston Pilot Schools/Horace Mann Network is a group of schools that 
envision education as a primary institution for achieving a more just, 
democratic, and equitable society. These schools have been given autonomy 
over: (1) budget; (2) staffing and hiring; (3) school schedule and calendar; (4) 
curriculum, instruction, and assessment; and (5) governance, which are 
essential elements in creating and sustaining successful schools. 

The Boston Pilot/Horace Mann Schools engage their students in rigorous 
and meaningful learning experiences. Their vision is to prepare students to 
become thoughtful, reflective, and creative individuals with the competence 
and habits of mind necessary to construct knowledge and understanding that 
facilitates engagement in learning. The Network believes that a primary 
purpose of education is to prepare students to access their talents and expertise 
in order to assume their roles in making important contributions in their local, 
state, and national communities. These schools are committed to: 

• Accountability — The School Quality Review process provides each 
school, as well as the district, with the opportunity to assess their 
progress every four to five years. 

• Equity — Data and documentation of accomplishments and patterns of 
achievement are disaggregated across race, gender, and socioeconomic 
status in order to identify effective practices that provide students with 
opportunities to access high levels of achievement. 

• Advocacy — Community organizing and political advocacy are 
employed with the goal of broadening constituencies that support Pilot 
Schools to ensure support and resources that are necessary for their 
continued success. 

• Family Engagement — Family participation, and respect, trust, and 
collaboration are encouraged to help create a successful teaching and 
learning experience for every student. 

• Leadership — There is support for school leaders, staff, students, and 
parents, with a focus on creating democratic schools through shared 
leadership. 

• Unifying Mission &Vision — School-wide practices and structures are 
driven by each school’s mission and vision. There is an ongoing and 
consistent effort to create a learning community that is reflective of a core 
set of beliefs and values about teaching, learning, and assessment. 
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• Teaching & Learning — Staff are engaged in: 

• Sharing and deprivatizing practice; 

• Developing a clearly articulated curriculum that integrates rigor of 
content, high-level thinking, and active use of knowledge; 

• Establishing explicit expectations for learning; 

• Challenging students to think about and reflect on their learning 
processes and strategies and to assess their progress over time; 

• Creating classroom communities that are caring, responsive to the 
needs of each student, and that stress the interdependence of teaching 
and learning. 

Principles and Practices 

• Teaching and learning reflect high expectations for every member of the 
school community. 

• Schools within the Network empower students to discover and develop 
their talents, strengths, purpose, and ideas. 

• Each school has a unifying mission and/or vision that is embedded in all 
aspects of the school, including curriculum, discipline, rituals and 
traditions, daily schedule/calendar, professional development, 
family/community engagement, and other school-related activities. 

• Those closest to the students are the policy and decision makers, which 
include teachers, administrators, support staff, parents/guardians, and 
students themselves. This requires democratic forms of school 
governance, shared leadership, and collaborative decision making. 

• The school is personalized so that teachers and students know each other 
well. 

• Professional development is characterized by reflection, collaboration, 
risk taking, and research-based innovation, and is an integral part of the 
daily schedule. 

• Learning is purposeful, authentic, and relevant — building student 
ownership and responsibility for learning. 

• Students are assessed in multiple ways, such as exhibitions and 
portfolios, in addition to standardized tests. They are expected to 
demonstrate their knowledge and competencies and its relevance in the 
community. 

• Families are partners in creating a high-performing school. 

• The school culture is centered around respect and trust. 

• All members of the school community share responsibility for student 
achievement. 
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Appendix E: School Lists 

BPS Schools 



Adams Elementary 
Agassiz Elementary 
Alighieri Elementary 
Bates Elementary 
Beethoven Elementary 
Blackstone Elementary 
Boston Adult Academy 
Bradley Elementary 
Brighton High 
Burke High 
Channing Elementary 
Charlestown High 
Chittick Elementary 
Clap Elementary 
Cleveland Middle 
Condon Elementary 
Conley Elementary 
Dearborn Middle 
Dever Elementary 
Dickerman Elementary 
Dorchester High 
East Boston High 
Edison Middle 
Edwards Middle 
Elihu Greenwood 
Eliot Elementary 
Ellis Elementary 
Emerson Elementary 
English High 
Everett Elementary 
Farragut Elementary 
Fifield Elementary 
Gardner Elementary 
Garfield Elementary 
Gavin Middle 
Grew Elementary 



Guild Elementary 
Hale Elementary 
Haley Elementary 
Hamilton Elementary 
Harvard/Kent 
Hennigan Elementary 
Hernandez K-8 
Higginson Elementary 
Holland Elementary 
Holmes Elementary 
Hurley Elementary 
Hyde Park High 
Irving Middle 
Jackson/Mann 
James Curley 
John F Kennedy 
Kenny Elementary 
Kilmer Elementary 
King Middle 
Lee Elementary 
Lewenberg Middle 
Lewis Middle 
Lyon Elementary 
Mary Curley Middle 
Madison Park High 
Manning Elementary 
Marshall Elementary 
Mather Elementary 
Mattahunt Elementary 
McCormack Middle 
McKay Elementary 
Mendell Elementary 
Mozart Elementary 
Murphy Elementary 
O’Donnell Elementary 
O’Hearn Elementary 



Exam Schools 

Boston Latin 
Latin Academy 
O’Bryant 



Ohrenberger Elementary 
Otis Elementary 
Patrick Kennedy 
Pauline Shaw 
Perkins Elementary 
Perry Elementary 
Philbrick Elementary 
Quincy Elementary 
Rogers Middle 
Roosevelt Elementary 
Russell Elementary 
Sarah Greenwood 
Snowden International 
South Boston High 
Stone Elementary 
Sumner Elementary 
Taft Middle 
Taylor Elementary 
Timilty Middle 
Tobin Elementary 
Trotter Elementary 
Tynan Elementary 
Umana/Barnes Middle 
Warren/Prescott 
West Roxbury High 
Wilson Middle 
Winship Elementary 
Winthrop Elementary 
Business Academy 
Excel High 
International High 
Mildred Avenue 
Monument High 
Odyssey High 
Public Service Academy 
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Schools Excluded from the Data Set 



Schools Exclusively Serving Students with Special Needs 

The following schools were not included in any analysis because they serve 
very specific populations that are not served by the other Boston Public 
Schools (for example, deaf or severe learning or behavioral needs). 

Carter Center 

Community Academy 

Expulsion Alternative School/Program 

Horace Mann 

McKinley Elementary 

McKinley Middle 

McKinley Technical 

McKinley Vocational 

Middle School Academy 

Young Adult Center 

Early Learning and Early Education Centers 

The following schools are Early Learning Centers (ELCs) and Early Education 
Centers (EECs). ELCs and EECs serve students in grades KO-1 in an extended 
day format. Given their focus on early childhood education and their extended 
day, these schools are qualitatively different from elementary schools. There are 
only six ELCs and EECs in the district, which makes them highly sought after. 
Students are assigned by lottery; there are long waiting lists; and there are no 
other public school options for students in grades KO and K1 who are not 
assigned to one of the six schools. Due to the qualitative differences between 
early education/early learning centers and elementary schools, students 
attending grades K2 and 1 in those schools were excluded from all analyses. 

Blue Hill Ave EEC 
East Boston EEC 
East Zone ELC 
Mattapan EEC 

North Zone ELC /Baldwin ELC (Pilot) 

West Zone ELC 

Newly Opened Pilot Schools 

In school year 2003-04, two new schools opened as Pilot Schools, Orchard 
Gardens K-8 Pilot School and Lilia G. Frederick Pilot Middle School (grades 
6-8). Their design teams were able to obtain Pilot status for these two 
schools, but although they received a Pilot designation, they were not granted 
complete autonomy over their school design. The constraints included: 

• Mandated enrollment of more than 700 students; other Pilot Schools 
enroll 500 students or less. 

• Mandated opening at full enrollment, as opposed to rolling out grades as 
did most of the other new schools that opened as Pilot Schools. 
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• Opening in a year with double-digit budget cuts, which resulted in more 
than 25% of these schools’ staffs being involuntarily assigned from the 
excess pool, meaning that a high proportion of the faculty did not choose 
to work in a school with Pilot status. 

• All faculty hires were made from August 2003, precluding adequate 
faculty planning time for the new schools. 

Because of their large size, involuntarily assigned faculty, and lack of 
faculty planning and understanding of autonomies, the two schools did not yet 
have full Pilot autonomies in the 2003-04 school year. Therefore, these two 
schools are not included in the outcomes analysis in the body of the report. 
However, outcomes for each individual Pilot School are included in Appendix 
C. In future years, as they have a chance to fully implement their autonomies, 
they will be included in analysis and reporting. 



50 Progress and Promise: Results from the Boston Pilot Schools 



www.cce.org 



Acknowledgments 

Report Authors 

RosannTung, Monique Ouimette, Leah Rugen 

Assisted by 

Vera Mouradian, Dongning Bai, Bob Frank, Dan French, Mary Doyle, 

Dawn Lewis, Vivian Unterweger, DaniaVazquez, Beatriz Zapater 

CCE Research Advisory Group 

Tony Dejesus, City University of New York-Hunter College 
Susan Moore Johnson, Harvard Graduate School of Education 
Miren Uriarte, University of Massachusetts -Boston and 
Mauricio Gaston Institute 

We wish to thank: 

all of the teachers, administrators, students, and families of the Boston Pilot 
Schools. In addition, we wish to thank the Boston Public Schools for providing CCE 
with raw data on student demographics, engagement, and performance, and the Boston 
Private Industry Council for the college enrollment data. Finally, we are grateful to BPS 

Superintendent Thomas Payzant and BPS Director of Research, Assessment, and 
Evaluation Maryellen Donahue for reviewing the report. 

We also wish to thank the following researchers who took the time to review 
drafts of the report: 

Priscilla Wohlstetter, University of Southern California-Rossier School 
of Education 

Patrick McQuillan, Boston College-Lynch School of Education 
Karen Hunter Quartz, UCLA Graduate School of Education and Information 
Studies -Institute for Democracy, Education, and Access (IDEA) 

Mike Nakkula, Harvard Graduate School of Education 

Karen Mitchell, SRI International -Center for Technology and Learning 

Anthony Boral, Millennium Pharmaceuticals 

This report was funded in part through generous 
contributions from: 

Anonymous Foundation 

Bill & Melinda Gates Foundation 

Boston Foundation 

Brinson Foundation 

Goldberg Family Foundation 

Washington Mutual 

William E. Schrafft and Bertha E. Schrafft Charitable Trust 



www.cce.org 



Progress and Promise: Results from the Boston Pilot Schools 5 1 



CCE Executive Director 

Dan French 

CCE Board of Directors 

Officers and Executive Committee 

Chairperson:Terry Herndon, Massachusetts Institute of Technology"' 
Treasurer: Irwin Blumer, Professor, Boston College"' 

Recorder: Joan Connolly, Superintendent, Malden Public Schools"' 

Members 

Jim Darr, Senior Fellow, Boston Foundation 
Avram Goldberg, President, AvCar Group, Boston"' 

Gregory Groover, Reverend, Charles Street AME Church, Boston 
Rob Hollister, Dean, Citizenship and Public Service, Tufts University 
Deborah Meier, Founding Principal, Mission Hill Elementary School, Boston 
Linda Nathan, Fleadmaster, Boston Arts Academy, Boston"' 

Sergio Paez, Director of Bilingual Education, Leominster Public Schools 
Paul Reville, Executive Director, Rennie Center for 
Education Policy and Research 
Jacqueline Rivers, Executive Director, MathPower 
Brian Straughter, Principal, Mission Hill Elementary School, Boston 
ChuckTurner, Boston City Councilor 
Bak Fun Wong, Principal, Quincy Upper School, Boston 

Emeritus Members 

Vito Perrone, Director of Teacher Education, Harvard Graduate School 
of Education 

Ted Sizer, Professor Emeritus, Brown University 



* Denotes member of the Executive Committee. 



52 Progress and Promise: Results from the Boston Pilot Schools 



www.cce.org 



